The present invention relates to the processing of pictures, including still images and video sequences. For convenience, the term “image” will be used herein to refer both to still images and to video sequences.
Images are represented in digital form as an assembly of picture elements, or pixels. Each pixel is represented by one or more component values. In the case of color images, for example, the picture information of each pixel may be represented by the values of three components R, G and B, corresponding to the primary colors red, green and blue. For any given pixel, the combination of those three colors at an intensity indicated by their respective component values reproduces the picture information of the pixel in question. Each component can take on any one of a predetermined set of values, such as the integers from 0 to 255. The universe of all possible combinations of the values of R, G and B is referred to as primary color space.
An image can be represented in other spaces as well. This is advantageous in that the representation of an image in spaces other than primary color space may require fewer bits of information, thereby reducing the bandwidth and/or time needed in order to transmit and/or store the image. For example, an image can be represented in so-called luminance-chrominance space in which each pixel is represented by a component indicative of the luminance, or brightness, of the pixel and two or more other components indicative of its color content. One such luminance-chrominance space is YUV space, wherein each pixel is represented by three component values referred to as Y, U and V, Y being the luminance component and U and V being the chrominance components. Another luminance-chrominance space is YIQ space wherein the component Y is, again, the luminance component and I and Q are components representing chrominance in another form. Other color spaces, such as HSV (hue, saturation, value), are also known.
Linear transformations readily convert an image represented in one space into an image represented in another. For example, conversion of an image from primary color space to YUV space is effectuated by way of the transformationY=x1R+y1G+z1BU=x2R+y2G+z2BV=x3R+y3G+z3Bwhere x1, x2, x3, y1, y2, y3, z1, z2 and z3 are predefined coefficients.
The display of an image on, for example, a video screen or in a color print document typically requires that the signal applied to the device representing the image be represented in primary color space. This is a consequence of the fact that the display device itself, such as a cathode ray tube or color printer, represents each pixel as a cluster of three primary color dots, each having an intensity and/or size determined by the respective component value. It turns out, however, that a primary color space representation of an image is typically not the most efficient way to represent an image. Thus, in a typical image communication system, for example, the image is scanned by a still or video camera in primary color space (such as RGB), is converted to a luminance-chrominance space (such as YUV) and is encoded using a standardized encoding algorithm such as one of the MPEG standards. The bits carrying the information content of the image as generated by the MPEG coder are transmitted to a receiver, at which point they are MPEG-decoded to recover the luminance-chrominance representation of the image. The latter is then converted back into primary color space and presented to a display.
A problem that arises in this context is that the components that represent the image in non-primary-color space, such as Y, U or V, may be modified from their original values at any of a number of points in the overall system for any of a number of reasons. These include, for example, the presence of communication errors, and operations such as filtering, noise reduction and various kinds of enhancements. As a consequence, when the image representation is converted back into primary color space for display, the values of one or more of the primary color space components may be different from what they were originally. The displayed image will thus be different in at least one of its characteristics, such as intensity, contrast or color.
A particularly troublesome effect in this regard occurs if, as the result of changes in the component values of the image during transmission or other processing, the component values that result upon conversation of the image back to primary color space fall outside of their allowed range. For example, in a typical image coding/communication environment, each component is represented as an 8-bit word, allowing for 28=256 possible bit combinations corresponding to the 256 decimal values 0-255. Depending on how Y, U or V got modified, the values of R, G and/or B realized upon the conversion back to primary color space may be outside of the allowed range, i.e., less than 0 or greater than 255. The display device, however, is only capable of representing R, G and B values that are within that range. The most straightforward way of addressing this is to change the value of R, G and/or B that is outside of the allowed range to the edge of the range, this being referred to as clipping. Thus R, G or B values that are less than 0 are made equal to 0, and values that are greater than 255 are made equal to 255. Although quite simple, this approach can result in significant degradation of the image, including color shifts and/or loss of contrast. Accordingly, more sophisticated approaches have been proposed. Such approaches may, for example, modify the value of not only the primary color space component (e.g., R, G or B) that was out of the allowed range, but even the values of components that are already within the allowed range—all with a view toward minimizing the image's degradation, i.e., with a view toward having the displayed image appear as much like the original as possible.